Prosoniq Products Software is an affiliated member of the
Under Cover Music Group GmbH & Co multimedia KG.
PROSONIQ PRODUCTS SOFTWARE GmbH
Hauptstrasse 24 * 69151 Neckargemuend
Germany.
Fax: (+49) 6223-9206-21
E-mail: prosoniq@u-connect.com, info@prosoniq.com
Preface
Dear sonicWORX User,
First of all, I would like to welcome you to sonicWORX Artist, the first high-end audio processing software for Apple Macintosh computers based on the new PROSONIQ Audio Rendering® technology.
Prosoniq Products is a German software company exclusively researching and developing new techniques for audio editing and digital signal processing. During the last five years we have developed new methods and algorithms that allow new and previously unthinkable sound manipulations giving you enhanced access to all specific features of a particular sound on your Apple Macintosh computer.
Based on the technology of Neural Network processing employed for sound evaluation and processing, traditional analysis methods like the FFT or related transforms including all the negative side-effects become almost obsolete in this software. Prosoniq's unique Multiple Component Feature Extraction provides relatively unlimited access to distinct spectral properties as well as to the phase relation and exact frequency of harmonics, formants and temporal sound developments.
Combined and grouped into different types of audio manipulation, sonicWORX combines traditional digital signal processing with new manipulation possibilities to give you utmost control over your audio recording. It offers all the DSP functions you expect from a standard signal processing software as well as many new functions you never thought possible at the time (e.g. removing reverb or adjusting basic voice characteristics) performed completely in the digital domain. What's more, sonicWORX' flexible plug-in concept makes it easily expandable growing as your needs require. Further developments from PROSONIQ or affiliated third party providers will furthermore greatly enhance the possibilities of your copy of sonicWORX.
If you are already acquainted with Digital Audio Editing you may skip the following section and read "Installation" first. Otherwise the following section may introduce you to the new concept of sound representation sonicWORX is based on and will answer at least some of the questions on "how does it work?".
We hope you will enjoy the new ways of creativity in designing new sounds that sonicWORX offers to you. If you have any suggestions or comments please do not hesitate to contact us via fax or e-mail.
The Prosoniq Products Software Team
What is a Neural Network?
The term "Neural Network" is a descriptive synonym for a data structure derived from simplified models of "real", that is organic, connected nerve fibres. Biologists as well as computer scientists have learned that "biological computers", like for example the human brain, have the ability to find and classify even insignificant patterns in large, unstructured data sets. A human is able to recognize the face of a person he/she knows in a large crowd of unknown people within a very short time, even though that person may be viewed under unfavorable conditions, say, with the face half-covered by a hat. The human perception is also able to recognize a single person's face even if the facial expression may vary significantly, thus making a simple comparison of images by numerical methods impossible.
The difference between a computer processor (like the one in your Macintosh computer) and a human brain can also be described in terms of systems architecture: the Macintosh processor is a (albeit very powerful) single processor that processes many different kinds of mathematical operations within a very short time, while your brain consists of many zillions of processors, each with a very limited set of operations and also very limited processing speed, but interwoven in a very, very complex manner yet not fully understood. This is why a single-processor-system can be a genius in numerical calculations, while a Neural Network (a set of interconnected processors) makes up for the shortcoming of not being numerically exact by having the ability to recognize patterns and generalize "rules" out of a set of examples.
The concept of Neural Networks in practical applications
During the last 20 years or so, computer scientists have developed a growing interest in such Neural Networks and various computers have been built to study and develop techniques and practical applications on this basis. Since developing computer hardware and specific Neural Processors is a very time and money-consuming topic, scientists soon came up with the idea of simulating Neural Networks completely in software using standard computer systems. The basic idea of employing the technique of Neural Networks in software is to use a (necessarily very powerful) computer processor to simulate a complex system of interconnected nerve cells (processors) and to study the behaviour of varying network structures to external stimuli.
There are a number of practical solutions, most of which can be found in today's computer science applications: image and speech recognition, optical character recognition, weather forecast, quality measurement as well as even computer games (chess) and many more. Artificial Neural Networks, although many million times less powerful than the human brain, can do an extraordinarily good job in identifying and recognizing consistent patterns in a seemingly chaotic set of data.
Neural Networks in Digital Signal Processing
No surprise that Neural Networks seem to also be of great use in processing audio signals. There are in fact two main tasks for a Neural Network applied to audio material:
Audio Signal Decomposition
Instruments, such as drums for instance, can be viewed as a consistent, repeating pattern with the same or similar pattern specifications for each instrument. A Neural Network of an arbitrary structure is able to identify the position and characteristics of a pattern (instrument) even if it appears in combination with other instruments playing at the same time. The Neural Network can find similarities between instruments in different combinations as it can identify the hand writing of a person and translate it into computer characters (this is called the ability of "Generalization").
Model Classification
After having isolated distinct patterns within the audio material, a specialized Neural Network can resolve similar patterns and formulate "structural rules" describing the probable momentary physical structure of the "virtual" instrument playing the respective piece of sound. This is quite the reverse procedure of "Physical Modeling", a process of sound generation by simulating the behaviour of real acoustic instruments that became very popular during the last two years. To establish this, the Neural Network first classifies the piece of sound (base model, class, subclass and singlet) and estimates the rendering parameter for reconstruction. In other words, sonicWORX tries to construct a virtual instrument that, if played, gives the analyzed piece of sound at a given moment. The more complex a sound is, the more instruments per second are required to represent a piece of that sound (and the more complex the instrument models become, respectively). To maintain seamless continuity, the instrument's physical structure is smoothly transformed (or 'morphed') between different models during reconstruction, allowing for a perfect phase and timing continuity. Several mathematical tricks usually applied to video rendering routines can be used to speed up sound decomposition and reconstruction as well as increment the accuracy of the process up to a stage, where about 8 transformations can be sequentially performed before audible side effects occur. This is because sonicWORX is adaptively resolving the sound until all components are classified and there is only a minimal amount of leakage in this process depending on the quality parameters adjustable with the "Preferences..." dialog.
On the importance of new analysis techniques
New transformation methods (as e.g. Prosoniq's MCFE) that provide an increased resolution of sound characteristics are the basis of advanced sound-resolving algorithms that make sonicWORX a powerful sound processing system. Neural Networks can be used to represent the spectral contents of a sound in a far more precise and (from the view of the human ear and auditory system) "natural" way than "pure technical" procedures as e.g. FFT and related transforms do (the FFT for example creates a spectrum with fixed grid points, that is, there are no "true" harmonics but only a set of sines representing different frequencies between the basis frequency of the viewed window and half the sampling rate that do not necessarily have anything to do with the sound's true spectral properties. True harmonics have to be found or resynthesized by interpolation, a process known not to be too exact. Furthermore, the FFT resolves well at high frequencies but poorly at the low end, quite the opposite to what our auditory system does. There are a number of work-arounds (pitch tracking, FFT-oversizing and zero-padding, interpolation, amongst many others), each of which introduces new trade-offs, but there is no final "perfect" solution for the problem).
Personal note on the use of FFT in this software
The popularity of the FFT in the field of audio DSP is obviously a result of (1) its age and (2) the amount of "musicality" often assigned to the sum-of-sines concept underlying the process. It might also be a result of its promotion by Universities. It really is a great tool for developing and designing filters and performing convolution, correlation and power spectrum estimation, but that's all there is to it. As far as sonicWORX is concerned, FFT techniques are used only in connection with necessarily (or due to computational savings) FFT-based algorithms such as complex filters and Transform Multiplication or solving special sets of equations. The use of FFT has been omitted where phase relations are critical or true harmonics are of interest. In these cases an advanced technique (MCFE) has been used. Please note that due to optimization the 68k Macintosh computers perform a quicker (and less exact) analysis and rendering that might not in all cases be superior to corresponding FFT based implementations, if ever possible.
Summary
To put it in other words: imagine the process of painting a picture. You usually have a "palette" of different colours and are creating the picture using combinations of these after (more or less) well considered thoughts on how to place which colour and how to draw the different lines. This is quite a similar procedure to what sonicWORX does when recombining a sound. It takes different instrument models (colours from a palette) and mixes and chains them together to form a complete audio recording (sequential drawing of lines to form a picture). As it is up to the painter's taste to modify or change certain aspects of his picture, you may freely determine the form and features of your audio recording by modifying its structural rules using the parameter editor.
A new analysis and rendering technique superior to conventional methods of analysis and resynthesis is being used, actually transforming the audio material into basic "physical instrument models" (the painter's "palette") and a reconstruction schedule (a recipe of how to compose audio from these instruments) rather than to a set of sines or other types of basis functions. By using sonicWORX' advanced DSP functions you are the "conductor" of your personal "virtual instrument" orchestra and can freely change timbre and performance of the orchestra inside your Macintosh.
Please understand that terms such as "virtual instrument" and "model" are used for illustrative purposes only. A more technical (and accurate) description would lead to techniques discussed in video/image rendering and to complicated mathematical descriptions of data sets as e.g. "Hidden Markov Models" and "Self-Organizing Maps", which would be of no illustrative use for understanding how sonicWORX actually handles your data. Note also that there is also no direct correspondence with the process of "Physical Modeling" used in sound synthesis there is far more correspondence with image rendering by which a digital image is generated from a set of descriptions than to such techniques.
The two main applications of Neural Network processing in sonicWORX
There are two main tasks that can be accomplished by a Neural Network in an audio application:
1. Decompose, generalize and reconstruct audio signals (see above) and apply parameter changes to the rules derived before reconstruction. This process can be used to produce new and previously unavailable sound effects in a superior quality.
2. Optimize a "conventional" DSP algorithm to optimally process audio signals to the requirements of an arbitrary task (e.g. optimize the compression scheme of an multi-band compressor to take into account the masking effect for different frequency bands to the auditory system thus giving an effect that is optimized to the listener's ear rather than to technical parameters). This application can be used to realize "conventional" DSP algorithms that are of superior quality due to a great amount of flexibility and adaptivity to a given signal.
Both of the above-mentioned features of Neural networks are used in sonicWORX. Please note that with sonicWORX Studio and especially sonicWORX Professional (also available from your local music store) you have access to a number of additional advanced signal processing algorithms that let you modify the acoustic properties of your audio recording.
Installation
Installing the sonicWORX demo software
• Copy all unpacked files of the sonicWORX demo archive into a directory of your choice on your harddisk. It is recommended to use an empty directory, the name of it doesn't matter.
Please take care that the sonicWORX program and the "sonicWORX PlugIns" Folder are in the same directory on your harddisk, otherwise sonicWORX will not find and fail to load the plug-in modules.
• Finally turn off virtual memory on your Macintosh computer. Virtual memory is enabled and disabled via the "Memory" control panel in your system folder. Although sonicWORX will run under virtual memory, drop-outs and artifacts may occur when playing back audio files. This is due to how virtual memory works. Please make sure that prior to recording or mastering important audio material virtual memory is disabled.
Macintosh 68k and PowerMacintosh machines
Differences between 68k and Power Macs
As you very well know there are two basic computer architectures available from Apple at this time. The older "68k machines" are based on Motorola's 680xx (frequently abbreviated as '68k') processors and the newer "PowerMacs" utilize an advanced RISC (Reduced Instruction Set Computer) processor which is many times faster than the 68k processor. We very much recommend the use of a Power Mac when working with sonicWORX since the processing speed is significantly higher on these machines. sonicWORX uses a technique called PROSONIQ Audio Rendering® which is almost as computationally intense as image and video animation rendering with many millions of calculations necessary to resolve or reproduce a single second of audio material.
sonicWORX comes as a 'FAT' application, meaning that it can used by both 68k and PPC (PowerMac) computers. 'FAT' also means that sonicWORX is a NATIVE PowerMac application, making it up to 20 times faster on average in these machines.
The editing functions of sonicWORX are the same for both computer types, however there is a difference in how the DSP plug-ins will work.
As mentioned previously, there are a huge amount of calculations necessary to resolve and recombine the structure of a piece of audio. Furthermore, neural network processing as such is a very calculation-intense topic no wonder there are computers that do neural network processing on hardware (so-called Neuro-Computers). Unfortunately, we do not have such a computer readily at hand and thus have to use the hardware available. In the case of 68k processors this is quite a difficult task the 68k models have (optimization and adaptation already included) only 1/300 of the power necessary to bring complete PROSONIQ Audio Rendering® to you in real-time. The 68k models need approximately 30 times more time to analyze and build audio material than the PowerMacs do. Due to optimization and by simplifying analysis and rendering procedures this number can be decreased to a speed factor of 10. But still this enhancement may cause computation times that are quite burdensome, especially for long audio files.
As far as the process of simplifying analysis and rendering procedures on the 68k Macs are concerned, we have chosen the least audible optimizations. However, slight differences may be audible when directly compared to files rendered on PowerMacs. Please note that you can also affect quality and timing behaviour via the "Preferences…" menu settings (s.b.).
68k Macintosh Computer with and without FPU
If you are a PowerMac user, you can skip the following section.
To use sonicWORX' DSP power you need to have an FPU installed in your computer (FPU: Floating Point Unit: a special co-processor for mathematical operations). Many 68k models already have such a co-processor (e.g. SE/30, Mac IIx, IIcx, IIci, Centris 650, Quadra 650, 700, 800, 900, 950). With these machines you should be able to use sonicWORX without problems.
If you own a Macintosh without FPU (e.g. Performa 475) you are able to start sonicWORX but you cannot use any of the sonicWORX plug-ins. In this unfortunate case you should consider buying an FPU expansion board available from your local Apple dealer which equips your computer with a hardware FPU. An FPU is needed because performing all mathematical operations in software would lead to days of waiting times instead of hours on the 68k machines. As already said, PowerMac users are not affected by this problem at all, since all PowerMacs have an FPU on board.
Please note that several technical reasons make the use of an FPU emulation software (like John Neill's "SoftwareFPU™") impossible. We strictly recommend the use of a hardware FPU expansion in any case.
Overview and Introduction
Introduction to sonicWORX editor
sonicWORX accepts audio files of the formats 'AIFF', 'AIFF-C' (uncompressed only) as well as SoundDesigner II and unformatted Raw files. It will read one and multi-channel files with a word length between 8 and 24 bit (demo version supports only 16bit files) and with any sampling rate from 8 to 50 kHz.
The sample editor and all DSP functions of sonicWORX are completely disk-based the samples don't therefore need to fit into memory (RAM) and may be as long as you want. All changes to the source material are written to a new file allowing one undo step if the result of the processing isn't satisfactory. Please note that sonicWORX nevertheless requires approx. 8 MBytes of RAM to function properly. This is because the plug-in modules need a huge amount of RAM (in the worst case up to 4 MByte) to store intermediate results and/or pieces of audio.
Please note that all RAM needed by the plug-ins is allocated dynamically. Every plug-in needs only about 30 kByte of RAM when not executing and allocates the required amount of RAM when processing is started.
sonicWORX is split up into two main components: the main program (named "sonicWORX Artist") which contains all sound editing features such as copying, cutting, displaying as well as all playback, record and file format conversion capabilities, and the sonicWORX plug-in modules (contained in the "sonicWORX Plug-In" folder) which carry the actual processing power of sonicWORX. This structure allows for easy updating and configuring of the DSP functions of the program.
There are two types of windows available in sonicWORX Artist:
• the wave editor window shows you an amplitude/time diagram of the audio material and lets you select portions of the file. You may also set input/output configurations for the various plug-ins and start the execution of any DSP function (Execute Button, see illustration).
• the parameter edit window displays the parameter set (all sliders and buttons of a particular algorithm) for the plug-in that is currently selected. You may also select a parameter set from a list of sets to modify or apply it to a piece of audio.
Every plug-in (just like a hardware multi-effects processor) has up to two in- and output channels (A and B) allowing you to process stereo sounds, generate mono output sounds from a stereo input sound or vice-versa, depending on the particular routing within the selected algorithm. Any of the combinations of two input and two output channels are possible, such as "Invert Phase" (one input, one output), "Vocoder" (two input, one output), "Virtual Room" (one input, two output) and "Normalize Gain" (two input, two output). The number of output and input channels available for a particular algorithm can be determined by looking at the wave editor window which shows pop-up menu buttons at the right side of the window corresponding to the internal routing of the algorithm. If there is only one pop-up button next to "Alg. Input" the algorithm you have selected has only one input.
Please note that you cannot change the internal routing scheme of the algorithms (as you cannot change the hardware of an effects processor) but you can assign different sound files to the different input and output channels (similar to the effects send and return routing you normally do with a mixing console). If an algorithm has only one input, there is no way to make it a two-input algorithm. Where channel correlation is critical we have therefore supplied two-input algorithms, otherwise you have to apply a specific algorithm twice, first to the left and then to the right channel of your audio recording.
Preferences...
This function opens a dialog box in which you can set parameters global to all of sonicWORX' edit and DSP functions.
General settings:
Beep after execute
If this checkbox is activated you will hear an acoustic signal when processing has finished. Turn this option on if you have a particularly slow machine and want to hear from the nearby café if your sound is done.
Save after execute
As an owner of a 68k Macintosh, you will learn to appreciate this option. If the option is enabled, the calculated sounds will be saved to disk after processing has finished. This is very useful if you let your Mac process large files overnight. This option is disabled in the sonicWORX Demo version.
The processed file will be stored under the name it was last saved. An existing version of the file will be overwritten, but note that you can recall the last version by selecting "Undo" from the Edit menu.
If the file hadn't been saved under a valid file name, you will be prompted for one when you leave the "Preferences..." dialog.
PROSONIQ Audio Rendering Set-up Parameters:
MCFE (Feature Extraction) Set-up
The "Multiple Component Feature Extraction" (MCFE) setup simply determines the quality of sound analysis by adjusting internal parameters of the MCFE routines. You may choose from two settings:
Resolve and classify all events tries to resolve all significant events from an audio file and classifies unresolved events by their nearest neighbour or the known instrument type that comes most close (classification is evaluated by basic model, class, subclass, and singlet). It offers the highest audio quality but may also take more time to process a particular audio file.
Exclude and bypass unresolved passes on unclassified events as "structural samples" shallowing the recursion depth needed for analysis. This option is the fastest though not the one with the best quality. Quality difference between PowerMacs and 68k-Macs may be more obvious if this option is selected.
PMG (Phase Model Generator) Set-up
The Phase Model Generator is part of the Rendering Engine keeping track of all phase relations between the true harmonic developments of a signal. In order not to "diffuse" the output signal, an exact phase representation per harmonic (or the parameters of a random process in the case of non-periodic signals) must be created since the phase relation between harmonics and the generation schedule of random and periodic signals is crucial to the temporal location of a specific event. Please note that this has nothing to do with the phase information obtained by applying a FFT but rather affects the true temporal development of the obtained per-point continuous spectrum which is one part of the Audio Rendering process..
You may choose from two settings:
Render all signal features tries to obtain a maximum phase coherence and an exact representation of the input signal. This is the slowest option but will also produce the best results.
Render key events and interpolate uses a simplified (but faster) Rendering scheme for sound reconstruction. Quality difference between PowerMacs and 68k-Macs may be more obvious if this option is selected.
You may change the quality settings of sonicWORX at any time, even during processing. If the settings are changed during processing, the new quality is adopted by interpolating so no audible transition will appear.
Not all plug-ins depend on the Preferences setting in their calculation speed. Plug-ins that are affected by the parameters set here are marked by a leading period (•) in their names.
The programmers make no warranty, express or implied, concerning the reliability of this product, and accept no responsibility for any damages caused by use or misuse of this program.
IN NO EVENT WILL PROSONIQ PRODUCTS SOFTWARE GmbH BE LIABLE FOR ANY DIRECT, INDIRECT SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES RESULTING FROM ANY DEFECT IN THE PRODUCT, INCLUDING LOST PROFITS, DAMAGE TO PROPERTY AND, TO THE EXTENT PERMITTED BY LAW, DAMAGE FOR PERSONAL INJURY, EVEN IF PROSONIQ HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
All features and specifications subject to change without notice.
All trademarks are property of their respective holders.
SoftwareFPU is a registered trademark of John Neill & Associated. ENSONIQ, ASR-10 and EPS 16-PLUS are registered trademarks of ENSONIQ Corp. SGI und Silicon Graphics are registered trademarks of Silicon Graphics Computer Systems, Inc. sonicWORX, PROSONIQ Audio Rendering and PROSONIQ PRODUCTS are registered trademarks of Stephan M. Sprenger/PROSONIQ PRODUCTS SOFTWARE GmbH. Digidesign, AudioMedia II and Sounddesigner are registered trademarks of Digidesign, Inc. EMU is a registered trademark of EMU Systems, Inc. Eternal Machine is a registered trademark of Klangkonzepte GmbH. Apple, Macintosh and Power Macintosh are registered trademarks of Apple Computer, Inc. EMS and Rehberg are registered trademarks of EMS Rehberg GmbH, Germany. All trademarks not explicitely listed here are property of their respective holders.
